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ABSTRACT 

This paper assesses the credibility of a single total 
instrument score and various logical sub-scores derived from a series 
of sximmative judgments about the quality of teaching performance. The 
objectives were to compare the generalizability of alternative 
Teacher Performance Assessment Instrument (TPAI) scores, to compare 
the dependability of decisions which could be made with the scores, 
and to compare the relationship of the scores with learner 
achievement. Measures were made of teacher performance using the 
revised version of the TPAI. Learner ability was assessed with the 
Group Assessment of Logical Thinking in order to equate classes. 
Learner achievement was assessed with the Middle Grades Integrated 
Process Skill Test. Results show that: (1) the aggregation of 
summative judgments used in the TPAI scoring can be a valid and 
reliable procedure; (2) intermediate levels of scoring such as the 
TPAI competencies are more desirable than total instrument scores; 
(3) the total is a more reliable, but le?s valid, indicator of 
effectiveness; and (4) validity and dependability coefficients are 
adequate evidence to support the validity and reliability of the 
competency scores. Caution should be ezcerised in inferring causality 
of these teacher behaviors or learner outcomes based on these 
results. (PN) 
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HOW MANY TEACHER PERFORMANCE CRITERIA SHOULD THERE BE? 

A query such as the one posed in the title has a simplistic 
ring to it. It implies that there is an answer such as 2,143 or 
2- More importantly, it begs the larger question "criteria for 
what?" In Georgia, the answer has been "criteria for initial 
certification" since the fall of 1 9 8 0 when performance based 
certification was initiated. Of course, other states have many 
purposes for the assessment systems that have been designed and/or 
implemented since 1980, In addition to Georgia, states which have 
pursued assessment for initial certification include Virginia, 
Mississippi and Florida. Florida, Tennessee and Texas have been 
notable among states which have attempted to differentiate 
teachers for merit pay or career ladders based on on-the-job 
assessment among other criteria. 

Assessment for merit pay may imply a different kind ot 
decision-making process than does assessment for initial 
certification. While certification suggests that there will be a 
determination that the candidate possesses a set of basic skills, 
each of which will be assessed, ^.arit pay might be based on a 
single decision about the quality of the teaching per f ormance (s) 
assessed. However, even merit decisions could be based on a 
number of skill areas, lest a satisfactory over-all score be 
earned with a strong perfornance in one area compensating for a 
weak performance in another. Initial policy in the Texas career 
ladder program reflected this kind of concern with the stipulation 
that "Instructional Strategies" be weighted twice that of other 
areas such as "Classroom Management." 
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The nature of the assessment instrument is a strong 
determinant in resolvinq the question "How many teacher 
performance criteria should there be?" Data collection systems 
such as the Florida Performance Measurement System or the 
assessment component of the Beginning Teacher Assistance Program 
in Virginia are quite complex and ^ to date, not we ] 1 -documented . 
A system of summative judgments, such as the Teacher Performance 
Assessment Instruments or the Teacher Assessment and Development 
System, produces a set of scores which can be aggregated in a 
number of ways which can reflect different needs in an assessment 
context. Capie (1983) has described a number of these alternative 
data aggregation possibilities and has speculated about their 
effects on assessment results. 

The bottom line in designing an assessment system is that the 
scores which are generated must be credible, whether they are 
derived from an event sampling procedure such as the F^P^M.S., 
from a time sampling procedure such as that used in Virginia, or 
from an aggregated checklist of summative judgments such as the 
T. P.A.I. Credibility of teacher performance measures has a number 
of dimensions ranging from common sense believabi li ty to 
psychometric concerns such as validity and reliability. (Capie, 
1985 ) He points out that concern for reliability must include 
both minimizing observer error and maximizing the ability to 
differentiate candidates from each other or from a criterion. 

Hambledon (1984) has described a number of aspects of the validity 
of cr i ter ion re fere need test scores . 



PURPOSE 

The purpoce of this paper is to assess the cred}l)ility of a 
single total instrument score and variour "logical" sub-scores 
which are derived from a series of summative judgments cjbout the 
quality of teaching performances. Specifically, the objectives 
were to compare the g e n e r a 1 i z ab i 1 i t y of alternative T,P,A,I. 
scores, to compare the dependability of decisions which could be 
made with the scores, and to compare the relationship of the 
scores with learner achievement, 

CONTEXT 

The Teacher Performance Assessment instruments (TPAI) were 
developed between 1976 and 1980 and were introduced for initial 
certification in Georgia beginning in 1980, Beginning in 1981, a 
revision of the T. P.A.I, was undertaken. Multiple purposes were 
served by the revision including expansion of instrument content, 
improving feedback to teachers, and improving the psychometric 
qualities of the instrument. The revised T. P.A.I, was used for 
certification purposes in September 1985. The final field test of 
the revised T. P.A.I, provided an opportunity to conduct this and 
other research related to instrument design and use . 

The site for the field test was selected in an effort to 
minimize problems which had threatened earlier T. P.A.I, validation 
efforts. Middle school science classes were selected since one 
concern during instrument development was the heavy reliance on 

teacher effectiveness research which was completed in limited 
settings (e.g. elementary mathematics and reading classes). Also, 
the science classes allowed for a common area of instruction 



across a]l classes and, consequently, a common post-test which 
helped reduce random variation associated with these factors and 
helped ensure the curriculum relevance of the post-test. Finally, 
the topic for the unit was science process skills, a set of high 
cognitive level outcomes which are much different than the 
objectives of previous T.P.A.I. validation efforts which were 
restricted to teacher-made tests addressing relatively low-level 
cogni t i ve outcomes . 

Teachers were provided a two week unit plan during a ninety 
minute meeting when the nature of the unit and the requirements of 
the research were explained. The unit consisted of ten lessons, 
each with specified objectives and an outline of activities. The 
equipment that was required for the lessons was relatively common 
and easy to find. The activities were somewhat open in that 
teachers could modify them to suit the needs of their particular 
situation. Each of the lessons required some individual or group 
activity. The lessons were expected to be somewhat demanding for 
teachers, a fact which would increase variance in teachers' 
T.P.A.I. scores. 
Instruments 

Measures were made of teacher performance using the revised 
version of the T.P.A.I. Learner ability was assessed with the 
Group Assessment of Logical Thinking (GALT) (Roadrangka, Padilla 
and Yeany, 1983) in order to equate classes. Learner achievement 
was assessed with the Middle Grades Integrated Process Skill Test 
(MGIPT) (Cronin and Padilla, 1986). 

The T.P.A.I. The revised version of the T.P.A.I. consists of 
eight teaching competencies each of which must be demonstrated 



before a teacher is granted certification. Each competency is 
defined by three or four indicator statements, makin^j a total of 
thirty indicators. Each indicator is defined ny four descriptors, 
making a total of 120 descriptors. The competencies are the oasic 
units for d e c i s i o n - ma k i n q with the T. P.A.I, since each must be 
demonstrated satisfactorily prior to certification. The logic of 
the relationship among descriptors, indicators and competencies 
has been confirmed in an extensive content validation study. 
(Cronin and Capie, 1935) A listing of the eight competencies and 
their thirty constituent indicators is displayed in Figure 1. 

I NSERT FIGURE 1 ABOUT HERE 
The more precise descriptor statements are the observation 
units which are used to derive the competency ocores. A sample 
indicator with its defining descriptors is displayed in Figure 2. 
Observers examine all of the data which may be relevant to a 
descriptor and then make a decision about the adequacy of the 
observed performance in the context ti*at is being assessed. The 
"relevant data" may be portfolio objectives for Indicator 1, tests 
and other assessment strategies for Indicators 5 and 6, or the 
entire observed lesson for Indicators 8-30. During a lesson, an 
observer will take notes to help recall important events from the 
lesson, but s/he will not be concerned with specific frequencies 
of certain behaviors. For example , the number of times that a 
teacher redirects off-task learners is not so important as whether 

or not the teacher attempts to redirect learners when necessary 
and whether or not these attempts were successful. Thus, the 
observer determines if there has been persistent off-task behavior 
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and, if so, determines how the teacher has dealt with it. These 
decisions, "acceptable" or "not acceptable," will be the scores 
for the descriptors in indicator 28. The score for the indicator 
is derived from the descriptor scores using procedures that are 
specified in the instrument. Notice that the sample indicator has 
provision fo^ giving the teacher credit for the indicator if there 
IS no persistent off-task behavior. The indicator scores 
generateo from the three observers' observation data are then 
aggregated to create the competency scores. 

I nsert figure 2 about here 

T_h e G A L T . All students were administered the Group 
Assessment of Logical Thinking (GALT) (Roadrangka, Yeany and 
Pddilla, 1983) as a pretest in an attempt to equate learner 
ability. GALT is a pencil and paper test consisting of 12 items 
that follows a double multiple choice format and requires 
approximately 30 minutes to administer. The test is designed to 
measure identif i-ration and control of variables,, probabilistic, 
proportional, correlational, combinatorial, and conservat ional 
reasoning. There were two items for each mode of reasoning. The 
authors report a Cronbach's alpha estimate of .85. 

The MGIPT . At the end of the unit the teachers al 1 
administered the same a post test (the Middle Grades Integrated 
Process Test, Cronbach a.lpha = .89) which was matched to the 
instruction, was composed of 40 multiple choice questions, and was 
provided by the researchers. 



and the corrolation between GALT and MGIPT {r=.62). For each 
learner the expected post-test score was subtracted from the 
observed post-cest score. This difference was considered to be a 
teacher effect on the learner. The mean of these "teacher 
effects" for each class was used as a teacher ef f ec+" i veness index. 
Thus, a variable reflecting class mean was available for use in 
subsequent analyses where classes were considered to be the 
sampling units. 

Validity Indices . Simple correlations were computed between 
each of the TPAI measures and the teacher effectiveness index 
which was considered the criterion variable in the study. Thus, 
there were 122 correlations computed: 92 descriptors^ 23 
indicators, 5 competencies and a total instrument score. In 
addition a number of regression models were tested using selected 
indicator or competency scores as independent variables and the 
teacher effectiveness as a dependent variable, 

Generalizabil i ty Analyses Generalizability theory was used 
to plan the analyses of the TPAI data. Four factors were 
identified as important sources of variation: teachers, individual 
observers, observer types and performance indicators. The four 
facet design, with individual observers nested within observer 
types, is arithmetically identical to a simpler three facet 
fully-crossed design with teachers, observer types and indicators 
as sources of variation. As a consequence, the simpler three 
facet model was used in the analyses of the competencies and the 
total instrument. Analyses of the indicators were completed with 
a two facet design. No generalizability analyses were completed 
for the individual descriptors. For each analysis, teachers were 
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considered to be the facet of differentiation and other facets 
were treated as random facets of generalization. Values of rho 
squared and phi ( lambda) were computed to assess the suitability of 
the scores for di f f erent i a t i nq teachers from each other and from 
the standard of having all indicators at or above the minimum 
level . 

Results 

Mean scores for the descriptors, indicators and competencies 
are included in Tables 1 and 2. Mean scores for descriptors could 
range from 0 to 1.00 to indicate the portion of observers who gave 
credit for each. Means for descriptors ranged from .23 
(Descriptor 10c, Displays create a pleasant atmosphere) and .33 
(Descriptor 29c, Learners are provided verbal feedback about 
acceptable behavior) to 1.00 (for a number of different 
descriptors ) . 

Insert Tables 1 and 2 about here 
Mean scores for indicators ranged from .38 (Indicator 24, 
Stimulates learner interest) to .95 (Indicator 13, Uses acceptable 
oral expression) . Mean scores for the competencies ranged from 
.62 (Competency 6, Demonstrates appropriate instructional methods 
and Competency 7, Maintains a positive learning environment) to 
.90 (Competency III, Demonstrates acceptable written and oral 
expression and knowledge of the subject) . 

The mean of the teacher effectiveness index was -.08, 
incJicating that, on average, the classes did about as well as 
their ability would have predicted. However, the variance was 

o 10 
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large (standard deviation=3 • 2) with one class scoring nearly nine 
points (out of 36 possible) lower than predicted and another class 
performing nearly nine points higher than predicted based on their 
abi i 1 1 ies • 

Validity coefficients for descriptors are displayed in Table 
3. They ranged from a low of -.26 (Descriptor 14a, Demonstrations 
and/or information presented to learners is accurate and 
up-to-date), to a high of .56 (Descriptor 10a, The classroom is 
free of litter) . Twenty-three of ninety-two (one-fourth) were 
statistically significant (p<.05) and fourteen were greater than 



Validity coefficients for indicators and competencies are 
displayed in Table 4. For indicators the values ranged from .03 
(Indicator 14, Demonstrates command of the school subject being 
taught) and .09 (Indicator 15, Gives explanations related to 
lesson content) to .53 (Indicator 10, Provides a physical 
environment that is conducive to learning). Eight of the twenty- 
three were statistically significant (p<.05) and four of these 
exceeded .30. Correlations be tween competency scores and the 
teacher effectiveness index were statistically significant for 
five of the six competencies, with the sole except i on bei ng 
Competency V (Communicates with Learners). 

Insert Table 4 about here 
The regression analyses of the indicators produced a number 
of meaningful models, one of which is summarized in Table 5. The 
four indicators which were used in the model accounted for 46.3 
percent of the variance in between class differences in 
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Insert Tables 2 and 3 about here 
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achievement. The indicators dealt with the physical learning 
environment (#10), assessing learner pro^jress (#11), usinq 
acceptable written expression (#12), and providincj feedback to 
learners (#18). Similar analyses with the competency scores as 
independent variables were less able to predict clas5:> differences 
in achievement. The three variable model which is summarized in 
Table 6 accounted for only twenty-two percent of tne between c^ass 
variation in learner achievement. The three competencies in the 
model relate to organizing time, space and materials (Competency 
4), demonstrating acceptable oral and written expression and 
knowledge of subject matter (Competency III), and communicating 
with learners (Competency V), These values contrast with the 
correlation of the total instrument score with the teacher 
effectiveness index, where r=.32. 

Insert Tables 5 and 6 about here" 
The results of the generalizaoility analyses of the 
indicators are summarized in Table 7 and the analyses of the 
competencies and total instrument in Table 8, Values for rho 
squared for the indicators ranged from 0 (Indicator 14, 
demonstrates command of the subject, and Indicator 21 , Uses 
instructional aids and mater ^Is) to .66 (Indicator 30, Manages 
disruptive behavior among learners). Values of phi(lamda) were 
substantially higher although Sc.veral were as low as ,4. Values 
of rho squared for the competencies were quite variable, ranging 
from .08 to .71 for competencies three and eight respectively. 
The basis for the low values can be seen in the variance 
components which show relatively low teacher effects in both cases 
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and a substantial teacheu Ly observer type interaction in the case 
of Competency VI. Values for phi (lamda) were substantially 
higher, however, with the lowest being .53 for Competency III, 
demonstrates acceptable written and oral expressioii. Rho squared 
for the instrument was .71. 

Tnsert Tables 7 a n S 8 about here" 

DISCUSSION 

A successfully operating assessment program such as the 
initial certification program in Georgia limits a researcher's 
access to the teachers who might be affected by it. Teachers who 
are involved in an assessment probably should and can not be 
saddled with the added pressures of being research subjects. 
Consequently, this research was limited to experienced teachers. 
Of course , there was val ae in bei ng able to inc lude an en tire set 
of teachers in this pilot study. The va r i abi 1 i ty of teaching 
performance may have been greater in this sort of pilot study than 
it is with highly motivated beginning teachers who are selecting 
an "optimum" context. This increased variance may contribute to 
increased reliability and to the possibility of finding 
significant relationships with the criterion variable(s). 
However, these benefits to reliability may be offset by the fact 
that the observers may have been error prone since they were not 
experienced with the new instrument. The cumulative effects of 

these cannot be known, of course, but possible liinitations should 
be acknowledged. 

In Georgia The principal iriterest of work such as this lies 
with its potential implications for instrument credibility, 
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particularly validity and reliability. Since decisions are made 
about competencies in Georgia, these score*:^ are of mo.:t pertinent. 
The fact that all but one competency was related to the 
effectiveness index can certainly be encouraging. Compared to the 
competencies, a smaller portion of indicators and descriptors were 
related to the effectiveness index and some correlations were 
negative. Nevertheless, even the proportion of significant 
(p<.05) correlations arriong tne descriptors (about 25%) was 
substantially above chance. 

The magnitude of these correlations can raise questions. One 
of these is "What expectations should there be for the magnitude 
of coefficients such as these?" Several years ago there was a 
suyggestion that correlations should be higher than .39 before 
they are considered noteworthy (Medley, 1977). However, Gage and 
has suggested that much more modest correlations are all that can 
be expected and that small portions of variance accounted for may 
be significance (1978). The possibility of attenuation by 
unreliability should be acknowledged, however. In the present 
study, for example, with rho squared values for competency scores 
ranging from .08 to .71, the "true" correlations with the 
effectiveness index might be considerably higher than the values 
that were reported (Range of r values is .21 to .41.) 

A second, related question involves the stability of the 
results in a variety of contexts. Expecting the same descriptors 
tc relate to teacher effectiveness in other studies is not 
realistic. Similarly, expecting the same indicators to relate to 
effectiveness may be unreasonable. After all, some of these will 
have occurred by chance. And, some of these specific behaviors 
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may not be equally applicable in each specific context. The 
expectation for every competency to relate the effectiveness may 
be more reasonable. Although these reliability estimates m this 
study are tentative; competencies as a set are more reliable than 
are the more specific, subordinant elements. Furthermore, because 
the competencies are aggregates of specific behaviuj_s, there is a 
likelihood that the aggregate will be related to effectiveness in 
various contexts even though every constituent descriptor will 
not . 

To some extent, these last points begin to bear on the theme 
of the paper: "How many performance criteria should there be?" 
There should be enough criteria to make the process credible. No 
teacher with a substantial weakness should be able to pass into a 
"high status" category. For example, no teacher who is weak in 
management should be able to be called meritorious or even 
minimally competent. Furthermore, each of these criteria must be 
cr ed i b 1 e - - w i t h be 1 i e v a b i 1 i ty , reliability and validity. These 
principles apply equally to assessment systems--TPAI , TADS, PMS,, 
TTAS, Virginia's or Tennessee's. 

The results are particularly pertinent to system of 
dichotomous, summative judgements such as the T.P.A.I. or TTAS. 
The results demonstrate that these types of jdugements can be used 
to make dependable decisions and the scores can be related to 
teacher effectiveness. The results also suqgest that the most 
desirable criterion level may an intermediate aggregation 
unit--larger than descriptors but less than a total instrument 
score. The total number of descriptors scored ''acceptable" was 
quite reliable and it accounted for approximately 10% of the 
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variation in effectiveness as it was defined. Without great 
sacrifices in phi (lambda) many competencies were as valid as the 
entire instrument in this regard. And, since the competencies do 
not measure the same instructional processes, having multiple 
criteria enhances the likelihood of better predictions of 
ach ievement • 

The regression analyses demonstrate the increased prediction 
with scores on three individual competencies predicting twice as 
much variance in teacher effectiveness as did the total. Selected 
individual indicators can predict half or more of the teacher 
variation effectiveness. However, caution must be used in this 
regard. With so many possible independent variables, there is the 
possibility of finding significant combinations which are not 
meaningful. The combination presented in Figure 8 may be a 
fortuitous combination which illustrates this ^oint or it may 
illustrate an additional "trap." Indicator 10 may reflect a 
dimension of planning. Indicator 12 may represent a "G" factor. 
Indicators 11 and 18 may represent two critical aspects of 
per f ormance--informally assessing learner progress and providing 
feedback. All of this may be true. However, none of these 
assertions may be true. And, certainly no directionality can be 
claimed in the relationship despite the logic that says that 
reaching the minimal even in both monitoring and providing 
feedback enhances achievement. 

A different, more comprehensive validation model would be 
required to support this assertion. Perhaps an attempt to 
preassess teachers and learners could be followed by an attempt to 
"engineer" improvements in the performances of selected teachers. 



ERLC 



- 15 - 



16 



Thus, if changes would be documented and associated with 
differences in learner performance, there could be a strong case 
for causality claims about T.P.A.I- behaviors influencing 
achievement as well as for a stronger case for the validity of the 
scores . 

Ttie current study does have a number of elements which would 
be desirable in the first engineering study of chat sort. The 
common post-test, common content, use of a complete set of 
teachers^ common post-test, and use of class as unit of analysis 
are all important elements in reducing some of the random 
variations in the study. Should such a controlled study produce 
"desirable" results, then a more realistic, longer term research 
effort might be made. 

CONCLUSION 

The results of this study show that the aggregation of 
summative judgements which is used in the scoring of the T.P.A.I. 
can be a valid and reliable procedure. The evidence suggests that 
intermediate levels of scoring such as the T.P.A.I. competencies 
are more desirable than total instrument scores. While the total 
is more reliable, it is a less valid indicator of ef f ect i veness • 
The results provide evidence to support the validity and 
reliability of competency scores as they are used in Georgia since 
validity and dependability coefficients were adequate. However, 
caution should be exercised in inferring causality of these 
teacher behaviors or learners outcomes based on these results, 
based on this or similar stud ies . 
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Surmciry of TPAI Orgamraticn 



Planning 



Observation 



Obserya t ^ en 



PLANS INSTRUCTION TO ACHIEVE SELECTED 
OBJECTIVES 

1. Specifies or selects learner objec- 
tives for lessons. 

2. Specifies or selects learning 
activities. 

3. Specifies or selects naterials and/or 
media . 

4. Plans activities and/or assignments which 
take Into account learner differences. 



IV. ORGANIZES TIME, SPACE, P;\TERIAL$, AND 
EQUIPMENT FOR INSTRUCTION 

8. Attends to routine t^sks 

9. Uses Instructional tirne efficiently. 
10. Provides a physical environrnent that is 

conducive to learning. 



VI. DEMONSTPJMES APPROPRIATE INSTRUCT iOhAL 
METHODS 

19. Uses Instructional ff>ethods acceptably. 

20. Matches Instruction to learners. 

21. Uses instructional aids and wterials 
during the lesson observed. 

22. irrpletnents activities in a logical 
Sequence. 



IL 



00 

I 



OBTAINS INFORMATION ABOUT THE NEEDS AND 
PROGRESS OF LEARNERS 

5. Specifies or selects procedures or 
materials for assessing learner per- 
fornunce on objectives, 

6 Uses systematic procedures to assess 
all learners. 



11. Assesses learner progrsss during the 
lesson observed- 



VII. MAINTAINS A POSITIVE LEARNING CLIMATE 

23. CoiTTTKjnicates persorval enthusiasm. 

24. Stimulates learner interest. 

25. Defnonstra tes warnjth and friendliness. 

26. Helps learners develop positive 
sel f-concepts . 



HI DEMONSTRATES ACCEPTABLE WRITTEN AND ORAL 
EXPRESSIOfi AND KNOWLEDGE OF THE SUBJECT 

7, Uses acceptable written expression. 



12. Uses acceptable written expression with 
1 earners . 

13. Uses acceptable oral expression 

14. DefTcns trates comnand of school subject 
be i ng taugh t . 



VIII. MAlNTAfHS APPROPRIATE aASSROOM BEHAVIOR 



Z7. Maintains learner involven^nt in 

instruction. 
28 Redirects learners who are off-tasic 

29. Cormunica tes clear expectations about 
behavior . 

30. ,Mar\ages disruptive behavior. 



COMMUNICATES WITH LEARNERS 

15. Gives explanations related to lesson content 

16. Clarifies explanations when learners mis- 
understand lesson content. 

17. Uses learner responses or questions regarding 
lesson content. 

18. Provides information to learners about their 
progress throughout the lesson. 



Figure 1. 
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COMPETENCY VIII: MAINTAINS APPROPRIATE CLASSROOM BEHAVIOR 
Indicator 28:_Red1rects learners who are off-task. 



Descriptors 

a. Non-verbal techniques are used to 
direct learners who are off-task. 



b. Verbal techniques are used to re- 
direct learners who are off-task. 



c. Learners who are off-task are effec- 
tively redirected. 

d. Techniques are used to maintain 
attention of learners who have been 
redirected. 



***0R*** 

5. No persistent off-task behavior Is 
observed. 



Figure 2. 
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Table 1 

Mean Performance Level for TPAI rescriptors 



Descriptors 

Indicator abed 



8 


.85 


.78 


1 .00 


- 58 




• ^ o 








10 


.98 


1 . 00 


. 23 


.98 


11 


1 .00 


.65 


.58 


.63 


12 


.93 


.98 


.98 


.98 


13 


1 .00 


1.00 


1 .00 


1 .00 


14 


.98 


1.00 


1 .00 


.90 


15 


.83 


.98 


.93 


.73 


16 


.63 


. 98 


. 95 


. 90 


17 




± . 00 


1 . 00 


. 40 


18 




ftft 

.00 


. 83 


. 75 


19 


.98 


.95 


. 58 


.73 


20 


.93 


.83 


.90 


.88 


21 


1 .00 


.88 


.98 


.35 


22 


.20 


.95 


.97 


.45 


23 


.88 


.Sd 


.78 


.73 


24 


.33 


.40 


.63 


.53 


25 


.98 


1.00 


.90 


.93 


26 


.85 


1.00 


.65 


. 75 


27 


.98 


.73 


.78 


.80 


28 


. 73 


.95 


.70 


.60 


29 


.95 


.85 


.33 


.95 


30 


.83 


.85 


.85 


.85 
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Table 2 



Mean Scores for TPAI Indicators and Competencies 



Competency Indicator Mean S.D. Mean S.D. 



IV .75 .21 

8 .67 .30 

9 .69 .34 
10 .88 .22 

II 11 .79 .30 .79 .30 

III .88 .11 

12 .84 .21 

13 .95 .17 

14 .86 .18 

V .72 .26 

15 .68 .34 

16 .88 .21 

17 .84 .23 

18 .48 .30 

VI .62 .27 

19 .52 .35 

20 .62 .32 

21 .77 .22 

22 .53 .33 

VII .62 .28 

23 . 58 . 34 

24 . 35 . 35 

25 .88 .21 

26 .65 .33 

VIII .66 .37 

27 .63 .33 

28 . 53 . 38 

29 .76 .33 

30 .71 .36 
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Table 3 

Correlation between Achievement and Mean Legal Tearn 

Descriptor Scores 



Descriptor 

Indicator abed 



Q 
O 








. Z Z 


9 


. 38* 


. 39** 


. 42 


. 1 3 


i\o 




o ^ * 

. zo 


. 1(^9 


.JO 


1 1 

J. ± 


1 


■ )Cf\J 


- 9 R 






• ^ u 


. 28* 


• XJ\J 


. 12 


13 


. 27* 


.21 


- . 18 


.00 


14 


- . 26 


- . 03 


- . 03 


. 08 


15 


.02 


. 22 


- . 08 


. 15 


16 


- . 06 


.25 


. 28* 


.05 


17 


.09 


.22 


.56** 


.15 


18 


-.01 


. 11 


.01 


.11 


19 


- . 18 


. 14 


.24 


.25 


20 


. 23 


-.12 


-.18 


.40** 


21 


.21 


.28* 


-.01 


.36* 


22 


. 12 


.21 


-.25 


-.07 


23 


.06 


.07 


.25 


.05 


24 


. 07 


-.04 


-.10 


.11 


25 


. 30* 


. 22 


. 15 


.02 


26 


. 31* 


.09 


.30* 


.15 


27 


. 12 


.22 


. 23 


.27* 


28 


.27* 


. 19 


.35* 


. 23 


29 


. 29* 


. 33* 


-.12 


. 14 


30 


.06 


.14 


.31* 


.22 
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Table 4 



Correlations Between TPAI Indicators and Competencies and 
Teacher Effectiveness Index 



Competency indicator Correlation 



IV 




4 9 * 




8 


. 13 




9 


. 34* 




10 


.53* 


II 


11 


. 22 


II I 




.31* 






. 36 * 




13 


. ] 5 




14 


.03 


V 




.21 




15 


.09 




16 


. 26* 




17 


. 19 




18 


.12 


VI 




, 29* 




19 


. 25 




20 


.24 




21 


.13 




22 


.15 


VII 




.27* 




23 


.18 




24 


.11 




25 


. 15 




26 


.31* 


VII I 




.33* 




27 


.27* 




28 


.28* 




29 


.29* 




30 


.25 
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Table 5 

Results of Regression Analyses Selected TPAI Indicators 



Source Type IV p AR R 

SS 

10 109.5 .0002 27.7 27.7 

11 28.8 .0368 2.6 30.3 

12 54.9 .0fc'50 9.4 39.7 
18 26.4 .0452 6.6 46.3 

Modpl 184.8 .0002 46.3 



ERIC 
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Table 6 



Regress i on 


Analyses of Selected Compe tenc les 
Effect! ven ess Index 


Agai nst 


Teacher 




Source 


Type IV 
Sum of Squares 


P 






r2 


Competency 4 


44 . 4 


. 030 


16 . 9 


16 


. 9 


Competency 3 


12 . 8 


. 217 


1.9 


18 


. 8 


Competency 5 


8.7 


.326 


3. 3 


21 


. 1 


Model 




.035 




21 


. 1 



ERIC 
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Table 7 



Reliabi')ity Coefficients and Variance Components 
for TPAI Indicators 



Indicator p"^ (j) ( \ ) T O TO 



Q 
O 


07 


A 7 


' XJ 


* kJkJ i. 


1 


Q 




• / D 


. XJ\J J 


' kJkJkJ 


1 S 9 


i k7 




• ^ z 




. kJkJkJ 


. Kjy ^ 








05 


006 

• XJXJ KJ 


.110 


12 


. 09 


. 36 


. 004 


.010 


. 124 


13 


.49 


. 54 


.014 


. 000 


.04 3 


14 


.00 


. 29 


. 000 


.000 


.134 


15 


. 54 


.76 


.063 


.000 


. 160 


16 


. 24 


.43 


.010 


. 000 


. 100 


17 


. 21 


.44 


.011 


.003 


.122 


18 


.11 


.11 


.010 


. 000 


.242 


19 


. 50 


.82 


.063 


.006 


. 186 


20 


.30 


.71 


.030 


. 000 


.210 


21 


.00 


.44 


.000 


.000 


.120 


22 


. 34 


.78 


.037 


.006 


.211 


23 


.44 


. 77 


.050 


.008 


. 192 


24 


.52 


.89 


.063 


.000 


.170 


25 


.25 


.43 


.011 


.002 


.098 


26 


.44 


.73 


.047 


.001 


. 182 


27 


.45 


. 73 


.049 


.016 


. 175 


28 


.60 


.84 


.085 


.000 


.168 


29 


.64 


. 76 


.069 


.000 


.116 


30 


.66 


.80 


.084 


.000 


.127 
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Table 8 

Reliability Coefficients and Variance Components for 
TPAI Competencies and Total Score 











variance 


components 






Competency 






T 


0 


I 


TO 


TI 


01 


TO I 


o 
J 


• K/O 


• D J 




.001 


.002 


.000 


.000 


. IC/lL/ J. 




4 


.49 


.75 


.024 


. 000 


.011 


.020 


.012 


.001 


.128 


5 


.33 


.71 


. 013 


. 000 


.031 


.041 


.011 


.001 


.115 


6 


. 10 


.79 


. 004 


.000 


.009 


.057 


.023 


.006 


. 145 


7 


.41 


.81 


. 013 


. 000 


.045 


.023 


025 


.003 


. 137 


8 


.71 


.86 


. 060 


.002 


.008 


.040 


.012 


.002 


. 107 


Total 


.71 


.92 


.021 


.000 


.021 


.017 


.015 


.002 


. 139 
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